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The principal concern of thiis paper is to facilitate a rapproche- 
ment of experimental and cori’elational methodology. While others have 
discussed various relationships betwreri experimental and correlational 
methods (see Burt, I9I+7 ^nd I966; Creasy, 1957 ; Bock. 196O; HcKeon, I965), 
the present unification, based explicitly on the work of Louis Guttman, 
seems not to have been adequately discussed. 

My specific aim is to show algebraically the major relationships 
among conventional types of product-moment correlation coefficients 
(includi^ng simple, multiple and canonical r's as well as partial and part 
coefficients) and the standard methcnds for analysis of variance (uni- 
variate and multivariate, with or vithout cevariates, fixed or randem 
effects). These relationships are shown through application of a rank 
r eductio n theor em which Guttman (l 9 ^i^) first presented within tiie context 
of factor analysis. In focusing on an algebraic system no explicit atten- 
tion is given to statistical inferential aspects of the respective methods. 
Nevertheless, it is hoped tliat this paper will contribute to a better 
understanding of certain inlerential procedures and their interrelation- 
ships. Restricting the discussion to algebra should facilitate computa- 
tions for these methods and an f asy-to-use computer program based on 
Guttman 's theorem is briefly described for such applications. 

Suppose is an N X p matrix which is associated with p 
measures on each of N entities and that it is reasonable to examine certain 
linear associations among the colrjtins. If the pairs of variabl :^s are 
approximately linearly related, it will often be fruitful to exeimine the 
^ TFroduct U^ vhicli here will be designated as V^. Few assumptions about 
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the nature of the respective variables are required In so far as the 
algebra of n*ului variate systems is concerned. Measures may be randoDi or 
fixed variables; they may be associated with qualitative cr qx:antitative 
responses; or they could be fallible or infallible, original or derived. 
Moreover, certain variables in a given system might be labeled indepondent, 
dependent or instrumental and the marginal or joint distributions mav have 
an immense variety of foms even if scatterplnts are takozi to be approxi- 
mately linear. Of course the inte rpre tive xises of the derived 
can never be guaranteed even ^^hen the variates under .‘^tudy do satisfy 
major assiuaptions for a formal iLethod of analysis, attempt is ij^ade in 
this paper to elaborate on appropriate uses of the statistics, or to 
specify vhen subject-matter specialists are apt to find them of inter- 
pretive value, despite the ultimate importance of such questions. 

The rank reduction theorem states that for a p x p symmetric, 
Gramian matrix of rank r, and £ x ^ veight matrix W (such that 
W' W is non-singular) , the matrix 

(1) Vi = Vo - Vo W (W' Vo W)-l W Vo 
is a residual matrix of rank £ - £, Guttman (1952) pointed out that the 
theorem can be repeatedly applied to successive residuals so that at the 
jth stage one can generate a £ x £ residual 

where Vj is of rank r - (£j^ d- + , , , + ) , 

k 

If there are k stages, then V, will necessarily be null and £. = r, 

j=l 

All must be distinct and each product (Wj ’ Wj ) should be non- 

singular. 
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My interest centers only indirectly on the rank-reduction 
aspects of the theorem; the main interest is to identify certain entries 
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of various members of (l) and (2) under differing specifications for 
measures in the initial matrix Uq. It will be assumed for convenience 
throughout this paper tnat colum.ns of are scaled to have means of zero, 
although this assvmiption nay be relaxed at later points. Also, for con- 
venience, it vill be assumed that is non-singuJar as veil as Gramianj 
this assumption is usually realistic whenever N>p, the latter being almost 
universally desirable in practice. The remainder of this paper is divided 
into three sections, on Correlational Kethods. V.xperimental Methods nn<\ a 
Computationa l Synthesis. 



Correlational M'^thods 



In this section I shail fii&t exaraii.e a special case of equation 
(1); next a special case of equation (2) is considered; finally, brief 
discussion is given with regard to some general factor analysis pi ob lams. 
Begin by partitioning by columj:!s: Uq - |X: Yj . X is ox' 

order K x s and Y is N x t where t = p - s. This leads to 



Vn - U, 






5 ^xx ^xy I 



. VJhile it was noted that in terms of algebra no 



yx ^yy-' 

restrictions arc required for the variables ii< as to type, it may 
initially be helpful to thinly of as a set of £ random variables, Vf, 
will usually be taken as a general varj ar.ce-covariance matrix although it 
will be useful at times to designate Vq = R, ^ P ^ p matrix of simple 
product -memo nt con e lat ions. 

Recalling (l), which states that K (W Vq W)'\” Vc, 

let the £x £ weight matrix bo d'^fined as follcvc; r; 



I. 

o 

0 



Ig is an 



identity matrix of order s, and 0 is a t x £ null matrix. The following 
identifications may be mado: 



o 
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(1) VqW is a P X s_ covariance matrix for the first £ variables in 
the system with respect to the entire set of variables. If 

Vq ~ R, then RW is clearly a rectangular matrix of correlations, 
V W may thus be written as » ^xxl nr i ^x ( , 

L.H N 

(2) W is the covariance matrix Vxx first s variables. 

If ! is replaces by where and normalizes columns 

of then is a simple correlation matrix for tl.e 

first s variables, 

(3) Vq (v;' Vq V;)"^ is a P x s matrix of estimated least squares 
regression ccefficients . This matrix maj' be written as 



where rows cf B„ ^ are sets of s estimated least 

y . X 



y-x 



o 
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squares regression coefficierts for predicting the t 
respective crit eria (rows). If R L.id W W :s 3 jplaced 

by R then rows of are stat:dardized regression co- 

efficients; the latter matrix li.ay be vTitten as 2 ^ ^xx* 

(^i) Vq V/ (W’ Vo W Vo ifc* the covariance mp.trix o.. the entire 

set of predicted portions of the variables. This matrix may 
be considered as an approx:' n.a lion to the initial. iriatrlx^ 

written v ' , Mote, however, that th 3 first s 

^ 1 V V i 

variables predict th€mneive:} perfectly so all el'^ments of 
Vq except Vy^ are eouivalent to the c rresponding entries in 
when using this particular W, Tf then each diagonal 

•A. 

entry of Vyy = Ry^. is a squared multiple correlation coefficient 

associated with s predictors for one of the respective £ 

^ -1 
criteria, Ryy nay be written as Ry^ R^y, 



“ 5 - 



(5) = Vq - is the residual matrix vhich in this case will be 



null except for Vy^ - The normalized version of the residual 



o 
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V/ V ss fv - V ) is the matrix of sth order partial cor- 

\yy) ^ yy yy^ - 

reiatioris between all pairs of t criteria. If s_ - 1, all 
t(t - l)/2 coefficients arc 1st order partials. 

(6) Given that = R, the "half-nomalict V* matrix - Ry^. is 



a rr.atrix of correlations. For exairiple, if I^^yy.)^^ ” 

which contains ^ complemoits of squared multiple ci^ndi hi t>n.« as 
its main diagonal entries, is multi pit oi on the right by the 



- 



diagonal rjormializinn' matrix D 



(i 



sme , ) 

\ 



, then any col^omn of 



coefficients in (R - R ) Ty, \ will contain sth order 

^ yy yy (i - smc.) 

part correl aticns--corre7atiojus lctv»?en an oi'iginal criterion 
variable and the error portions of the ^ criteria with respect 
to the set of s r^redictors, 

(?) Canonical correlations between the sets of predictors and ^ 

criteria may be found as the positive square roots c-f the 

- 1 

eigenvalues of either the product Vyy Vyy , or in the 
ir.ctric of Vyy (see Dempster , 1968) , The same roots are associ- 

A -1 

ated -with Ryy K,y. See Meredith (196^) for a discussion of the 
algebra of canonical correlations under corrections for un- 
reliability of the variables, 

-1 d 

(8) Figenvecters (right-hand) of V , here called q*, define 

yy yy j 

the canonical variates for the t criteria with respect to the 

set of s predictors. The t dimensional column characteristic 

; -1 V 

vectors q^ may be ccjnputed es q. = G,r'y\ v, where G is the 

j (j-^’yj y^ 

set of (unit-length) columns of eigenvectors of V , y\ is 

y J y 

the diagonal matrix of reciprocal square roots of eigenvalues 

5 
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01 * V. and each V:: is an ci:,en vector of the s.’/TTJTiecric matri:: 

J J 

“1 ^ “1 

y\ Gr* V 0,, y\ . (Most algorithms for f^^iaratlng roots 
y ^ j y 

and vectors require yyrriTiiRfcric matrioes.) Tne set of canonical 

” n ~ 

variates for the s predictors can be found by using . 



and proceeding analo^.ously—tnns reversing the roles of the 
predictors and the criteria. 



Paragraphs ntJTibercd (9) - (l6) are aiiaj.ogous to those nuinbei'ed 

(l) - (8) except that equation (2) of Guttnian's theorem js now cous5dered, 

this leading to partial statistics of various ^inds most of which are 

direct, albeit complex, counterp-arts of the statisti.es already identified. 

Equation (2) states that V. =7. , - V. , W. (V . ' V, , W.)*^ K.' V. 

^ ^ ' J J-1 J-1 J ^ 0 0-1 0 0 0-1 

— . —I 

haoh matrix W. shall be taken to be of the simple fonn ^ ^ where s is 

J ■ 1 

“d I 

2 i 

J 

the diFiCnsionality of the matrix ; thus W. is of order p x s.. It will 

ij’ 0 ^ -0 

k 

oe convenient here to assume that “ ®i so that the set of p 

0=1 ^ 

variables in can be considered as a set of s = v s^ t ,,, + 
predictors (X) which is adjoined to t = p - s criteria ('/), J\3rther, let 

J* 

s* be Sj' for a particular j'$k. 

0=1 ^ 

Given the above spicci f icatiens , the following identifications 
may be made: 

(9) ^ P ^ £ j rectangular matrix of par*‘ial covariances. 

Analogous to in paragraph (l), Wj is just a vertical 

slice of latter being a matrix which is described in 

paragraph ( 13 ) below, 

( 10 ) Vj^ ^j“l Ej X sj matrix of £*th«order partial 

O 
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covariances for the set of predictors at the j’th stage. If 
col^unns of W. are scaled to normalize columns of V. , V/., 

J d " d 

as in then at stage Jy ^j»-l ^nj * 

of (s* - s ,)th order partial correlations among thxe s., 

-y ~ j 

predictors, 

(11) V. . W. (W.* V. , W.) is a p X s^ matrix of partial re- 

d"rj j J-lJ “ “*j 

gression coefficients. In general, the sj i predictors at the 
j*th stage are error portions of a particular set of predictors, 

viz., those portions of £ . , variables which are not linearly 

J 

predictable from the preceding ^ ' variables. Thus, the 

lower t X s . , portion of V . » , V . i (VC is a 

matrix each row of which contains semi-partial regression co- 
efficients for predicting a sint,le criterion from certain 
resid uals associated with tne £ ^ i criteria. 

(12) W. (VC* VC' V. T may be viewed as an approxi- 

d d d"-^ o d d“*^ 

-A 

mation Vj tt the niatrix with which it is associated. 

-A 

At stage j*, ^j'-i properly regarded as a set of partial 

covariances among the portions of £j , predictors which are not 

linearly predictable from the preceding s* - £ . 1 predictor 

variables and the remaining residuals in the system. If the 

A 

initial = R, then diagonal entries in 

contain squared multiple - partial correlations--tliose (error) 

portions of the respective ^ criteria which can be predicted, 

using a multiple linear equation, from the errors associated 

with the s.i variables as described abeve. Given that 
k 

ll “ the sura of all k such squai'ed multiple partial 
j=l J 

coefficients for a particular criterion variable is necessarily 
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equal to the squared multiple correlation coefficient asso- 
ciated with the entire set of £ predictors. If 

= $2 = t,, = s^ = 1 so that \i = £, then principal diagonal 

A 

entries in R . , - are squares of conventional semi-partial cor- 
0 ” -i- 

relations. It should be noted that values of a set of k semi- 
partial or multiple-partial correla.ti. ns (or covariances) 
are con ditional on a particular ordering of the k sets of 
predictors unless tl;e k sets are mutually orthogoiial, 

Rozebooixi (1966) is an excellent source on this variety of 
statistics. 



(13) V. = is a p X p matrix of partial covariances 

also. At the j'th stage of application of equation (2), 

V.t is null in its first s* rows and colunms. Tlie lover t x t 
J *“ 

portion is a pai'tial covariance matrix for the 

J 

(s* - s-t)th order residuals of the t criteria vith respect to 
~ J 

the portions of the s^, predictors which are orthogoncl bo che 

preceding ^ predictors* If Vq = R at the onset, then 

the diagonal entries of ^ contain proportions of variance 

0 

of the respective criteria which remain to be predicted after 
employing s predictors in a linear prediction system. As one 
moves from the j*- 1 to the j’th stage, equation (2} states 
that increments of error variability for each criterion variable 
must be reduced by tlie amount of the scoa\*ed multiple-partial 
correlation described in paragraph (12\ The normalized form 
of cither V/_.\ or R, ^ contains semi-partial correlations 



pairs of t criteria 
order error portions of the set of Sj 1 predictors constant. 



among the pairs of ^ criteria after holding the (£* - Sjt)th 
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(14) If = R at the onset then the ''half-noiinalizec^*' version of 

the t X t r^atrix , contains semi -part correlations. 

For example, if H(yy) . , is multiplied only on the ri^]ib by 
its nonnalizing diagonal, then a colu mn in the product matrix 
contains correlations between an - s.,)th order error portltDii 
of a i)e,rticular criterion variabre and the set of t - 1 
criterion residuals which cannot be linearly predicbed from e 
(s* - s^)th order errors of the s.| predictors at stage j'. 

(15) Canonical partial correiations between a particular set of 
(s* - s-i)th order res- iuals for s., predictors and the asso- 

— "J 

ciated set of criterion residuals can be obtained as in para- 
graph (7): such canonicals are the positive square roots of 

the eigenvalues of a matrix of the form \ V. . 

j ' ^yy)j , 

Sucn canonicals are indeed general since they subsume prac- 
tically all of the preceding correlation coefficients as 
special cases, 

(16) Eigenvectors of \ V, . define the canonical variates 

(rOji 

at the j'th stage for the (error portions of) criterion 
variables with respect to (s* - r.,)th order error portions of a 



particular set of s . , predictors. Such canonical variates may 

“J 

also be generated from a syirmetric raatrix as described in 
paragrapl'i (8), 

The foregoing correlational analysis is based on equations (l) 
and (2) of Guttman's ranh reduction theorem where the vreight matrices 
have particularly simple form. Much of what was presented has been 
’^available" for over fifty years* In addition to the synthetic aspects 
of the present elaboration, its chief virtue may be to automate the 

O 
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computation of practically all of the preceding statistics through sti'aight- 
forvrard application of a simple theorem^ a computer program for which may 
be readily developed for any digital computer. 

If the T/j matrices are allowed to range moi'e broadly, an immense 
variety of other methods can be identified with respect to equations Cl) 
and (2), For instance, starting with observed variables in U^, where 
Vq = Uo^ Uo, if the set of k weight matrices Wj (each of the order 
p X Sj) are allowed to range over all possible distinct arrays, all types 
of c omponent anal ysis can be defined. If each is a distinct eigen- 
vector which corresponds to the initial (of rank k = r), then equation 
( 2 ) specifies the exhaustive set of principal c cmpcnents of V^. If 
Vq = R, then principal component analysis is defined as it is typically 
employed. Of course, the usual apru'oach to PC analysis, and common 
factor analy.sis as vrell, docs not involve a priori specification of ’weigiit 
matrices; derived factors can, neverti^eless , be define d in terms of the 
rank reduction theorem. 

If Vq is replaced by a Gramian variance-covariance matrix of the 

2 2 

’^common portions" of variables, say, D^ , where is a p x p 

diagonal matrix of uniqu'^ness variances, tnen any set of (Thurstonian) 
common factors can be generated for a given V - D^, using a specific set 

of weight matrices V/p, Guttman (1952) discusses certain 

aspects of the latter variety of applications in some detail. 

Other varieties cf "factor analysis" can also be associated with 
the rank-reduction theorem, nay be taken, for example, as the 
Vtur iance-covariance ir^atrix of the "images" of the initial p variables 
(see Guttman 1953) or seme "reproduced" portion of tliis matrix using a 
limited number of factors (e.g, see Karris, 19^2 ) , 
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For any such approach to factor analysis, based on either (l) or 
(2), Guttman noted that one could, by choosing weight matrices appropriately, 
avoid completely the task of "rotation'* -- or factor transformation. 

Indeed, Guttman (l952) argued that interpretahle factors based on weight 
matrices chosen a priori, presumably using a particular scientific justi- 
fication, are apt to be most compelling. While space limitations pre- 
clude a discussion of this argument, it is clear that there are special 
virtues of a direct analysis of a complex of variables using Guttman *s 
rank reduction tlieoren.. Those who are interested in a modern-day version 
of such "Procrustes" factoring are encouraged to study the monumental work 
of Karl doreskog (I969jl970)on analysis of covariance structures. 

Equations (l) and (2) can be shown to be a sufficient basis for 
defining other methods as well, but ve shall emit discussion of such 
possibilities in order to proceed with applications associated v;ith 
analysis of variance. 

Expe rimental Methods 

In this section Guttman 's rank reduction tSieor^n is used as a 
vehicle for generating sums of squares and cross-products viiich forni the 
basis of computations for a vdde variety of inferential tests in the 
analysis of variance. It vdll seen tint the algebra of correlational 
metliods, as presented in tl;e preceding section, subsumes tue algebra of 
practically all of the standard forms of analysis of variance including 
those involving multiple dependent variables and covariates for fixed or 
randem effects. Moreover, many (balanced) incomplete designs may be in- 
cluded v/ithin this paradign^. 

It should be noted again in passing that questions oi "meaning- 
fulness" of sample statistics are not being considered here. Any 

O 
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interpretative use of derived statistics involves distributional theory as 
well as assumptions about the nature of data for any given study. Need- 
less to say, such factors should be given careful attention if any of the 
extant methods are to be employed in data analysis. Depending on v;hat the 
investigator determines to be a reasonable choice of independent and 
dependent variables in a study, the procedures of this paper can be readily 
applied using a simple computer program as discussed in the final section. 

Again it is convenient to consider an N x p matrix which is 
partitioned into a set of s_ predictors X (usually called independent 
variables) and a sot of ^ = p s_ criteria Y (usually called dependent 



variables). Thus, U = []] X ; Y j and ~ 



^xx \ ^xy 

V ' V 
_ ''yx » ''xx 



also for convenience, colmns of will be taken to have zero means, which 
implies that is a sum of products matrix 02 a covariance matrix. Tue 
rank reduction theorem of equations (l) or (2) will be used with weight 



matrices W or V^2> of tiie same simple form 

employed in the foregoing section. 

The reader may prefer to think of dependent variables Y as con- 
tlnacus random variibles while X may best be vievred as a set of fixed or 
random variables. In particular, let independent variables be partitioned 
into a set of contrasts (vuicli may correspond to "planned cemparisons", 
"main effects" or "bloc];ing variables"), contrasts Xg (which may be 
associated v/itli classical "interactions" of two or more main effects) and 
randen variables (v;iiich ray be associated with "covariates" as in 
covariance analysis). FurliiCr, each of the subsets X^, Xg and will in 

turn be vertically p^^rtil oned to alio* or designs of relativel^^ more 



o 




s . 

rd 



as v;ere 
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complexity as when, say, three main effects (X^p, studied, 

using four interactions ^B13> ^B 23 > ^8123^ covariates 

(^ci> ^02)’ 

The plan for the remainder of this section is to emanerate in 
paragraph form how the rank reduction theorer. may be employed to generate 
sums of squares and cross-products for the major types of designs (one 
; actor, factorial or nested each being with or without covariates) for 
either univarjate analysii .'=1 (AN'^jVA/ANCOVA) or rnulti variat6 analysis 
(MAKOVA /MAI^COVA ) . Attention v i 1 1 be i n i t j ally x’ e s i* J ^ o d to p_KJ^o£on^ 
factorial designs. After discuss:, .g forrrial algebra for major design 
categories several issues vhich x>ertain to applications are briefly ex- 
amined. Reference is made to exariples in the Appendix to facilitate the 



exposition , 



One Factor (m)AKO VA 



In the case of conventional one factor ANOVA or MAN0VA> as in the 
fully randomized design, equation (l) of Guttinan's theorem is sufficient 
to generate desired product matrices. The matrix X, or X., of U is taken 
as a set of £ = J - 1 contrast vectors when there are J groups Y is a set 
of t response variables; t ■= 1 for univariate (ANOVA) studies or t >j for 
multivariate (liANOVA) studies. Example 1 in the Appendix, which is also 
examined in Pruzek (l97l), depicts contrast vectors for a simple four 
group MAKOVA where t = 2, The Guttman theorem may be used as 



V = V - V W (W'V W)"\rv for w = 
1 O O ' o ' o 



I 

S 

0 - 



; the lower right t x t 



= V 



A 

V. 



portion of tlie three respective matrices may be written V/ \ 

\ yy / yy yy 

c-r, in more conventional teminolCcY, E - T - B, where T is the total sums 
of squares and produces matrix, B is the between or among matrix of sums of 
Q luares and proc'.ucts; E is tl.e residual or within matrix* Univariate 
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analyses correspond to these being 1 /. 1 matrices ^ i,e. scalars. The 
particular scaling of columns of Y are irrelevant with respect to statistical 
tests since all standard test results are known to be invariant with respect 
to linear transfoiinations of dependent variables. 

It should also be noted ;he s^ contrast vectors may be associated 
with individual degrees of freedc»c; if the contrast vectors correspond to 
particularly "meaningful" comparisons of various combinations of groups, 
then equation (2) designates the usual planned co mpa r ison approach where 
= s ^2 ~ ••• “ ^J-1 " contrast vectors are not mutually orthog- 

onal, then difficulties will typically arise in interpretation of the 
planTiCd COL pari sons for a particular ordering of contrasts. Orthogonal 
planned comparisons are therefore apt to be most useful. The reader may 
wish to satisfy himself that the basic computatior i for many post - hoc tests 
may also be carried out within the present context. 

One Factor (M)AKCCVA 

The fundamental distinguisliing feature of a covariance analysis 
is that the independent variable set includes one or more (antecedent) 
random variables in addition to fixed variables. For one-factor analysis 
of covariance, X is comprised of and matrices as in X = (X^ j X^). In 
this case two stages of equation (2) are required to generate matrices for 
tne analysis of covariance. First, for a set of covariates, use 



^( 1 ) = 




and note that the lower t x t portions of the three product 



product matrices in = V 



’ V W,,, (V/^vV corrsspond to 

o (1) (1) o (1/ (1) o 



the equat-^on = Vyy - Vyy or, in standard terminology, T'*^' = T - T 

/V 

where T represents the portion of the total variation for t variables 
which is linearly associated with the covariates. Kiiile one might naturally 
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be led to find using equation (2) where the weight matrix selects con- 
trast vectors in X, covariance analysis is not appropriately done this way 
since to do so would be to improperly specify regression coefficients. 



Rather one generates 



Vq (WiVo Wi) 



V^i Vq where 




for = J - 1, the number of contrast vectors in and fojlovs by finding 
Vg = Vi - Wg (V;^Vi Wg) where = 

the lower ^ x ^ portion oi' is of the form E = T - B, it foilows that the 

/V A 

lov/er ^ X t portion of is » = E - E where E is the portion of the within 
groups variation linearly predictable from the set of covariates. One 
factor M/UICOVA thus requires T* and E* as v;ell as B* = X* •• E’* as its sums 
of products matrices for any of the standard MATIOVA tests with appropriate 
degrees of freedom. See Bock and Haggard (I968, p. I30) for a further dis- 
cussion of this method. For an illustration consider Appendix example 1 
where for purposes of illustration the 1st response variable is taken as a 
covariate witij respect to the second response variable. 

Factorial De sign (M) ATJOVA 

Con^putations for factorial designs may be generated by using X^ 
and Xg matrices which are comprised of aggregates of contrast vectors. Any 
particular aggregate of contrasts may be viewed as specifying comparisons 
among defined groups of (vector) observations. Example 2 in t:ie Appendix 
includes an X matrix which might be employed for a 2 x 3 factorial design. 
Reference wi3.l be made to this example in the following general discussion. 

If all subgroups associated with a factorial design are of equal 
size» then contrast vectors for different main or interaction effects will 
be mutually orthogonal. Tiiis implies that the ordering of effects is 



o 




0 



i.e. 



'(!)■ 



Since 



lb 
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inconsequential with respect to the magnitudes of the derived sums of 
products matrices or the associated test statistics. If one desires a set 
of k-j^ main effects, he simply specifies = [ as a set 

of appropriate contrast vectors, as in example 2. Each aggregate of con- 
trasts corresponds to a row partitioning of (vector) observations in 
the dependent variable set Y, Successive applications of Guttman's theorem, 
following equation (2), generate V^, ••• well as the approxima- 

/V ✓N 

tions Vq, ««• V]^ ; the lower right t x t portions of these products 
contain useful matrices for (M)AKOVA computations. The set 



(yy)2’ 

A 



and the final residual V 



(yy)kj^ ~ '^(yy) * ^'^(yyOi ^ 



+ ^(yy)kj^^ simply the total sums of products for factors (main effects) 
and, ultimately, a residual matrix. If interaction effects are desired for 
any combination of effects, these are obtained using applications of 



equation (2) with respect to = 



,, X, 



] 



for k^ the 
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|~ ^B12’ ^B 13 ’ "Bkg 

number of interaction effects. Each aggregate of interaction contrast 
vectors may be found using elementwise products of entries in contrast 
vectors associated with a particular combination of main effects. The 
necessary sums of products matrices now will be Vyy, ..., 

V/ , V/ V/ , and the final residual V/ ^ . . , 

(yy)^i ’ (yy)^i + ^2 + ^2 

The degrees of freedom for a particular effect are always equal to the 
number of vectors in a particular aggregate; this is true for main effects 
or interactions. See example 2 below and tue analyses of this 2x3 design 
by Mori i son (1^*67) as veil as by Fru^ek If, as is being assumed 

here, the subsets of contrasts in X are mutually orthogonal, then any t x t 
matrix of the fonn will be the same, vhetiier ccxnputed as 

“ \ "j <"j ''o 

It; 
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may be especially UvSeful in applications where only a ocraputer program, for 
product -moment coefficients is available. See Bock and Haggard (1968) for 
an excellent review of the statistical considerations for I'iANCOVA applica- 
tions. Also see Mendenhall (1968) for more discussion of the linear models 
approach to ANOVA, and Graybill (I96I) for technical details. 

Factorial Design (M)AIICOVA 

This class of methods simply adds a special wrinkle to the 
methods reviewed in the preceding paragraph, Tiie major null hypothesis with 
which either AHCOVA or MAIiCOVA is concerned is that particular effe/’ts 
null when dependent variables are linearly adjusted to the values they 
ostensibly would have if all relevant groups of entities were ec, li valent to 
one another on the antecedent variables. Since Bock and Haggard (I968) 
present the general matrix approach to ^^ANCOVA, it is necessary here only 
to express their results in the terminology of this paper. 

Once all of the sums of products matrices for effects of the pre- 
ceding paragraph are computed, as well as a t x t residual matrix E, then 
one proceeds to generate a “corrected'^ residual matrix E = E - E where E 
represents the portion of the residual variation which may be attributed 
to the set of covariates in and a special "corrected” total matrix for 
each effect of interest, A separate subtotal matrix must be ccxr.puted for 
each effect in order properly to generate the minuend with rospect to which 
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E* is the subtrahend for producing the corrected effects matrix of sums of 

vs ^ 

products. That is, T* = T. - T. v?here T. represents the portion of the 
0 J J 0 

vS 

matrix Tj = E + which is linearly predictable from the set of co- 

^ "^j * required for tlie jth effect in 

KRUGOVA. Bock & Haggard point out that any of the standard multivariate 
tests may be applied to the matrices using the usual decrees 

Q Df freedCHn for effects and error degrees of freedom d.f.^ - k^ for the 



variates in X^, Then V 
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number of covariatea. 

Mested Design MAMOVA/MAMCQVA 

The concept of nesting has already been tacitly discussed in the 
foregoing paragraphs of this section, Whenever one replicates a particular 
experimental combination in order to generate a group of (vector) observa- 
tions, the observations within the group are properly regarded as nested. 
If, as in orthogonal designs, one nests the same number of observations in 
every experimental group then the final residual matrix, say E, may be 
regarded as an aggregate of nested effects assoeiated vJ th entity vaviati<>n 
within groups, 

This concept may be generalized, however, as when a set of 



factors for a design :.s hierarchically arranged. The computations for 
nested designs are entirely straightforward with respect to the results 
already given for general factorial designs, Tiie major distinguishing 
feature of computations for ne,. ed designs is that interaction effects 
have no meaning so ths.t depending on the "tier" of a factor in a hierarchical 
design, computation of the effect matrix involves pooling contrast vectors 
which for factorial designs had been associated with interactions and with 
main effect contrast vectors. As an example if three factors 1, 2 and 3 
are hierarchically arranged as 1, 2 within 1, 3 within 2 within 1, then 
contrast vectors for the three effects of this design w'ould require three 
Wj matrices using equation (2) which selected j "| , p X^2i ^B12^ ’ 

and p X 

for details of the discussion of caxputations and interpretations of 
effects for these designs, Mulcivariate extensions of his discussion are 
irinediate , 



AO * ! ^ooo ' "1 contrast matrix X, See Winer (I962) 

Ao ; I 




Covariance aralysis for nested designs could be done along the 
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same lines as discussed in the preceding paragraph were an investigator to 
regard this as useful. 

Incomplete Design MANOVA/MAI^COVA 

The limited objective of this section is to present an approach 
to the analysis of balanced complete designs under the rubric, fractional 
factorials. These designs formally include latin square designs and 
balanced incomplete block designs so to cover fractional factoi'ials in 
detail is to be reasonably comprehensive. It should be noted, hoviever, 
that there are incomplete designs, particularly unbal.flnced i ote 

ups, which do not lend themselves to a matrix-based analysis. 

Fractional factorials may be characterized as factorials from 
which selected experimental combinations have been systematically deleted 
to allow study of a relatively large number of factors using a relatively 
small number of experimental units. The deletions amount to taking rows 
away from the design rriatrix X to yield X* so that contrasts 'e ^'ain in X* 
(i.e. means of columns of X* are zero). Thus, equation (2) of this paper 
suffices to generate sums of products matrices which are commonly used in 
statistical tests as i»i the paragraph on factorittl designs above. For the 
Most used variety of fractional factorials, the 2 ^ designs, the V/j matrix 





0 




” 0 “ 


for the jth effect is simply a vector of the form 




= 


1 




0 




0 











the value of unity corresponds to the jth contrast vector in X* -- the jth 
effect. See Mendenhall (1968) for a discussion of 2^ ^ designs within the 
context of linear models and McLean (19^^) for an example which, because 
the author gives raw observatjons, is readily susceptible to analysis 
using t)ie approach suggested here. 

As with nested designs, covariance analysis can be conducted for 
O hese inccmplete designs using the methods of the paragraph on M/J^*C0VA. 
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Although the foregoing discussion is abbreviated and largely 
restricted to computational methods^ I do not wish to convey the impression 
that I condone a cavalier approach to choosing either independent or 
dependent vari'ables. Some commentary regarding issues and general prob- 
lems of experimental analysis may therefore be useful. 

Regarding the approach generally, which is based on Gutlman's 
rank reduction theorem, two references should be cited. Suits (1957) 
discusjed the issue of dummy variables in regression pointing out that 
two or three different regression approaches may be taken to the specifi- 
cation of independent variables for the analysis of variance. It was 
made clear that the contrasts approach is not uniquely appropriate despite 
the fact that it lends itself to a particularly simple fCnalysis for 
general applications and facilitates a comparison of experimental and cor- 
relational analyses of data. The second reference is Beaton (1964) which 
includes a discussion of a ”Gweep’^ operator for matrix calculus which is 
in certain respects closely related to the present approach. Dempster 
(1968) discusses Beaton's operators in some detail and applie^' them in the 
analysis of multivariate data. 

Another issue is that of selecting dependent variables for 
analysis. If an investigator is primarily inte.^,sted in statistical tests, 
especially those which are likely to require less stringent assumptions 
about data than conventional parametric tests, he may study the approach 
of Gabriel and Sen (1968). These authors present a test statistic based 
on ranked scores which may be derived using the algebra of the present 
paper where each column of observations in Y is taken as a set of ranks. 
Their chi square statistic is shown formally to specialize to the Kruskal- 
^Wallis statistic for ANOVA using ranks as veil as the Wileexon rank scores 

ERIC St. 
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The more general problem of selecting metrics for dependent 
variables continues to be studied. In the case of experimental data, 
strong arguments have been made (see Tukey and Wilk, I 966 ) for transfomi- 
ations which reduce the portion of variance for non-additivity of a 
variable. That is, one should choose bne metric for a variable in such a 
fashion that interaction effects will be relatively small in relation to 
main effects. Pruzek (l97l) discusses the general issue of choosing 
dependent variables further with som^ attention to the issues of variable 
unreliability and choices among transformations. Various plots of residual 
values (see Wilk and Gnanadesikan, I 968 ) become essential when examining in 
detail the appropriateness of a model for analysis j in tlie next section a 
computational approach is outlined for coi:iputing residuals for various sums 
of products matrices of the present paper. Textbooks such as Mendenhall 
(1968), Winer (1962) and Hicks ( 1964 ) should be consulted lor discussion of 
issues relating to the selection of independent variables in AI 1 OVA, and 
Bock (1963) and Bock and Haggard (1968) are strongly recommended for dis- 
cussion of the matrix approach to MAWOVA and MANCOVA. 

Compu ^J^ion al Syn_t hesis 

The major objective of this paper has been to unify classical 
correlational and experimoutal approaches to th.e aucalysis of data witli 
respect to the algebra of computation , It has been sliovn that a single 
theorem, originally due to Guttman (1944, 195-) > can be seen to provide a 
vehicle for drawing th.ese well-known classes of methods together. Refer- 
ences have been given throughout the paper "ro facilitate the reader’s 
furtl'ier study cf these relationships with respect to a wide variety of 
issues and problems which are ccrr^only ei.countered in scientific research, 

A computer program, has been written to generate all of tlie 
Q matrices of equations (l) and (2) for Guttman 's rank reduction theorem. 
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The program jls presefitly written to read in data in the form of a crosn- 
products (or covariance, or correlation) matrix and a series of weight 
matrices, Wj, which are to be specified according to the user. (The pro- 
gram is being modified to read in ’Mata" matrices of the form and to 

allow "on-line" specification of a sequence of weight matrices; these 
improvements should substantially facilitate instructional uses of the 
present approach besides permitting more convenient computation of various 
residuals, as is noted below), 

Oitputs of the p^rogram are most conveniently labelled in the 
language of factor analysis; however, the reader may wish to make reference 
to either of the first two sections of this paper to help interpret entries 
of the respective matrices. Equation (l) corresponds simply to a special 
case of equation (2) so no further attention is needed for the simpler 
equation. 



Given and •••) follovdng matrices are included 

in the output ; 

This is a pattern matrix for the primaries in general. 

(b) W .V. , v; . for j = 1, ..., k 

This is the correlation matrix of t!ie "primaries" in tiie 
lan^pjage of factor analysis. 

(c) V. V! . (W .V. , VJ for j 1, k 

' '' j-1 nj '• nj , 1-1 ’ 

This is tho structure matrix for the "primaries"; entries nre 



(d) 



o 
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properly regarded as regression coefficients . 

V. , w . (W V , W )-lw , V. , for j =1, k 

flJ nj j-1 nj nj J-i 

^ A 

Tills is an approximation, V. ,, to the matrix which is 

based on variables vliicli are selected by K . , 

J 
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(«) Vj - Vj.i - (B'.Vj.j -I " '. ■••■ I' 

This is a residual matrix associated with the comparison of 



V4 1 and Tha lover x t portion of tnis matrix 

j—j. 



may 



be of special interest in applications if matrices are 

chosen in the fashion of this paper. 

(f) D„ Vj Dn for all J =1, .... 

This is a matrix of partial correlations associated with 

any pairs of vai'iables for which the diagonal elements of 

V- exceed zero. 

J 

The set of jr-roducts (a), (b) and (c) may also be generated without 

scaling V/. to normalize V-: i , as when raw score regression coefficients are 
J 

desired . 



; cainple 1 of the Appendix gives the initial ” L ^ J ^ 

t = 2 one factor MANOVA as well as the associated V and tlie derived 
~ o 

matrices E = T - eacli of order 2x2. Columns of X have been chosen to 
correspond with orthogonal polynomials since doing the analysis in this way 
allows quick i-and check. 1.3 of the results; = X^X is diagonal as is 
. (The reader may wish to satisfy himself that in this context 
orthogonal polynomial coefficients offer great potential for generci] appli- 
cations of factorial design MAIKOVA regardless of whether factor "levels” 
are quantitatively related to one another.) The required (corrected) sums 
of squares for one -facte r MAKCOVA are also given for this example follow- 
ing the strategy of paragraph (2) in the second section. Eor this illustra- 
tion the first dependent variable is used as a covariate with respect to 
the second. 

The next example in the Appendix gives the raw data for y and evo 
Iternate design .natiices, labelled and X^, for a 2 x 3 factorial design 

23 
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MAIKOVA* The data are taken originally from Morrison and are also included 
in Pruzek {l97l). The matrix is scaled to correlation metric and this 
Vq is used to derive sums of products matrices for testing the effects for 
Rows (SEXES), Colvimns (DRUGS) and Interaction (SEXES & DRUGS), 

It should be clear from these examples that the algebra is 
equally appropriate for many more general applications of either correlation 
or regression analysis, 

Finally, it should be helpful to indicate how one might generate 
matrices of residuals given matrices of the form Vj , or and the 

algebra of Guttman's theorem. Such quantities are easily derived using 
matrix equations. 

The classical least squares estimates of factor (or component) 
scores in factor analysis may be expressed in present terms as 

% = (W'Vo ''0'^ 

Howc/er, within che context of this paper, where estimates of dependent or 

criterion scores Y are of special interest, one can use the equation 

^ -1 -1 
Y = Y V V V V 
^ yy yx ''xx'^xy 

which may also be written 

^ '^yy^(yy)j 

Thus, to generate residue Is at any stage one needs only to compute 

-1 -1 

^{yy)3^(yy)^ shown to be equivalent to j r (Vj » 

for tlie differences between estiricates at stage j^-1 and stage y» 
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^xannle 1 

Given the data in the ur^ne ; novtion of Table 1, the svins of 
nroducts mat;;rices, of orde’^ 2x2, vere derived for erro', total and betv;een 
sources of variation. These matrices are desirjneted I?, T and B and are n re- 
sented in the lo:;ev portion of Table 1, Tor covariance analysis, u si nft 
the 1st d splendent variable as a covariate for the second, three corrected 
suras of squares are T* -- 28 - (Uof/l7h, = 3 - (tj)^/30 and 2- = I8.80 
- 2.!)7 = 16.33. 



TIxamole 2 

Table 2 includes the vav7 scores x'or Y as vrell as the de /let ion 

scores Y (either of v;hich ^;ill f.-^nerate the desired effects matrices belov;) 

and tv:o possible design (X) matrices for frene;at5n.^ suns of 'oducts for a 

2x') yAil(NA \7here t 2, The data are taken from ;*orri son (lSi68). Table B 

includes the V matrix scaled in correlation metric as uelL as the associa- 
o 

tec. sums of -n-^oducts matrices from rhich statistical tests may be carried 
out (see ’^ru^ek (l97l) for an analysis of these data vrhich follov:s these 
linos), T’.e matrix v^as used f o ’ this narticulav' Vq, 
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Table 1 



Independent and Dependent Variable Values and Sums of Products 
For a t = 2 IvlAlTOVA with n = 2 for Eac:h of t Groups 



X /■ 



"311 




” 2 " 


3 11 




5 


1 -1 -3 




2 -1 


1 -1 -3 




6 -1 


-1 -1 3 




1 0 


-1 -1 3 




-5 -1 


-3 1 -1 




-5 -2 


-3 1 -1 




-7 -1 


- 









30 


4 “ 






40 “ 




- l44 


36 " 


E = 


4 


3 


, T = 


ho 


28 


, B : = 


36 


25 




- 


- 




- 








- 
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Table 2 
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An Example of Data and Tvo Possible Design Matrices 
for a 2 X 3 Orthogonal MANOVA where p = 2 and n =* 4 




“ 5 


6 ' 


5 


i* 


9 


9 


7 


6 


7 


6 


7 


7 


9 


12 


6 


8 


21 


15 


i4 


11 


17 


12 


12 


10 


7 


10 


6 


6 


9 


7 


8 


10 


10 


13 


8 


7 


7 


6 


6 


9 


i6 


12 


l4 


9 


14 


8 




( 1 


1 0 


1 0 ) 


( 1 


0 1 


0 1 ) 


( 1 


-1 -1 


.1 - 1 ) 


(-1 


1 0 


-1 0 ) 


(-1 


0 1 


0 - 1 ) 


(-1 


-1 -1 


1 1 ) 



( 1 


1 1 


1 1 ) 


( 1 


0 -2 


C - 2 ) 


( 1 


-1 1 ! -1 1 ) 

1 

1 


(-1 


1 1 


-1 - 1 ) 


(-1 


0 -2 


0 2 ) 


(•1 


-1 1 


1 - 1 ) 
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Table 3 

Certain Matrix Pi'oducts^ Associated with 
Morrison's 2x3 l-MOVA with t = 2 



(rescaled in correlation r.etric) 



l.CO 



.00 


1.00 








,00 


.50 


,1.00 






.00 


.00 


.00 


1.00 




.00 


.00 


.00 


.50 


l.CO 


.o 4 o 


-.765 


-.716 


-.173 


-.148 


.060 


-.443 


- . 258 


-.406 


-.295 



1.00 

. 71 !+ 1.00 



yy 



(yy)i 



(:y)2 



(yy )3 



Vo. 



(yy )3 



1.00 .714 1 


.002 .002 ■ 




“ .733 .355 " 




b 

0 
I 


.230 .279 ' 


- 




+ 










l.OCO J 


. C 04 




.198 




— 1 
1 


.622 _ 



For syranetric r-.atrices, only upper-right portions arc given 
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Errata for Pruzek's ^*Un1f Ication of f^ethodology ' 



Page 

5 

6 
7 

13 & 16 
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Correction 
Para. (8) , 1 , 4. 
Line 1 . 

Para . (11), last 1 . 

Top p. 13, top and 
middle of p. 16. 

1. 3. 



Should read "computed as^.= G v." 

j y j j 

Should read "of and,," 

Should read "with the ^ ^ ' predictors.'* 

Contrast matrices should be vertically 
p artitioned , as in I - ^A3^‘ 

Should read "considerations for MANOVA.." 
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Appendix 



1, 4, Should read "balanced m:omplete designs,." 

A -I A 

Last equation & ^ ^ v 

sentence which follows \yyj^l 

"V; ''(yy)j' ^(yy)j' 

to be equivalent to ^o'^(yy)j ’ '-1 



Ex . 2, 1. 1 & 2 delete words between "raw scores for Y,." 

and "..and two possible design (X) matrices" 



Table 3 footnote Should read ",, only triangular portions are 

given. " 

.. also upper portion of U 'U should read 

T.oo ° " 

■00 1.00 (etc.) 
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